Skip to content

HADOOP 17290 to 17811. ABFS Backports#3253

Closed
sumangala-patki wants to merge 3 commits intoapache:branch-3.3from
sumangala17:backport-HADOOP-17290to17811
Closed

HADOOP 17290 to 17811. ABFS Backports#3253
sumangala-patki wants to merge 3 commits intoapache:branch-3.3from
sumangala17:backport-HADOOP-17290to17811

Conversation

@sumangala-patki
Copy link
Contributor

@sumangala-patki sumangala-patki commented Aug 2, 2021

sumangala-patki and others added 3 commits August 2, 2021 11:48
@sumangala-patki @sumangala17
HADOOP-17290. ABFS: Add Identifiers to Client Request Header (apache#…
dd4e052 
…2520)

Contributed by Sumangala Patki.

(cherry picked from commit 35570e4)
@sumangala-patki @sumangala17
HADOOP-17765. ABFS: Use Unique File Paths in Tests. (apache#3153)
87f2918 
Contributed by Sumangala Patki

(cherry picked from commit 10ba4cc)
@brianloss @sumangala17
HADOOP-17811: ABFS ExponentialRetryPolicy doesn't pick up configurati…
1ff850a 
…on values (apache#3221)

Contributed by Brian Loss.

(cherry picked from commit 1d03c69)
@hadoop-yetus
Copy link

hadoop-yetus commented Aug 2, 2021

🎊 +1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 12m 58s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 1s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 markdownlint 0m 0s markdownlint was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 44 new or modified test files.
_ branch-3.3 Compile Tests _
+1 💚 mvninstall 37m 20s branch-3.3 passed
+1 💚 compile 0m 33s branch-3.3 passed
+1 💚 checkstyle 0m 25s branch-3.3 passed
+1 💚 mvnsite 0m 39s branch-3.3 passed
+1 💚 javadoc 0m 29s branch-3.3 passed
+1 💚 spotbugs 1m 4s branch-3.3 passed
+1 💚 shadedclient 19m 22s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 💚 mvninstall 0m 37s the patch passed
+1 💚 compile 0m 33s the patch passed
+1 💚 javac 0m 33s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 checkstyle 0m 22s the patch passed
+1 💚 mvnsite 0m 34s the patch passed
+1 💚 javadoc 0m 22s the patch passed
+1 💚 spotbugs 1m 20s the patch passed
+1 💚 shadedclient 20m 39s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 unit 2m 14s hadoop-azure in the patch passed.
+1 💚 asflicense 0m 37s The patch does not generate ASF License warnings.
101m 2s
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3253/1/artifact/out/Dockerfile
GITHUB PR #3253
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell markdownlint
uname Linux e96fdeec2246 4.15.0-128-generic #131-Ubuntu SMP Wed Dec 9 06:57:35 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision branch-3.3 / 1ff850a
Default Java Private Build-1.8.0_292-8u292-b10-0ubuntu1~18.04-b10
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3253/1/testReport/
Max. process+thread count 549 (vs. ulimit of 5500)
modules C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3253/1/console
versions git=2.17.1 maven=3.6.0 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

@sumangala-patki
Copy link
Contributor Author

sumangala-patki commented Aug 2, 2021

TEST RESULTS:

HNS Account Location: East US 2
NonHNS Account Location: East US 2, Central US

AppendBlob-HNS-OAuth

[ERROR] Failures: 
[ERROR]   ITestAbfsStreamStatistics.testAbfsStreamOps:140->Assert.assertTrue:42->Assert.fail:89 The actual value of 99 was not equal to the expected value
[ERROR] Errors: 
[ERROR]   ITestAzureBlobFileSystemLease.testTwoWritersCreateAppendNoInfiniteLease:178->twoWriters:166 » IO
[ERROR] Tests run: 558, Failures: 1, Errors: 1, Skipped: 98
[ERROR] Errors: 
[ERROR]   ITestAbfsFileSystemContractSecureDistCp>AbstractContractDistCpTest.testDistCpWithIterator:635 » TestTimedOut 
[ERROR] Tests run: 271, Failures: 0, Errors: 1, Skipped: 76

HNS-OAuth

[ERROR] Failures: 
[ERROR]   TestAbfsClientThrottlingAnalyzer.testManySuccessAndErrorsAndWaiting:171->fuzzyValidate:49 The actual value 9 is not within the expected range: [5.60, 8.40].
[ERROR] Tests run: 103, Failures: 1, Errors: 0, Skipped: 1
[WARNING] Tests run: 558, Failures: 0, Errors: 0, Skipped: 98
[ERROR] Errors: 
[ERROR]   ITestAbfsFileSystemContractSecureDistCp>AbstractContractDistCpTest.testDistCpWithIterator:635 » TestTimedOut
[ERROR] Tests run: 271, Failures: 0, Errors: 1, Skipped: 52

HNS-SharedKey

[WARNING] Tests run: 103, Failures: 0, Errors: 0, Skipped: 2
[ERROR] Failures: 
[ERROR]   ITestAbfsRestOperationException.testCustomTokenFetchRetryCount:93->testWithDifferentCustomTokenFetchRetry:122->Assert.assertTrue:42->Assert.fail:89 Number of token fetch retries (4) done, does not match with fs.azure.custom.token.fetch.retry.count configured (0)
[ERROR] Tests run: 558, Failures: 1, Errors: 0, Skipped: 54
[ERROR] Errors: 
[ERROR]   ITestAbfsFileSystemContractDistCp>AbstractContractDistCpTest.testDistCpWithIterator:635 » TestTimedOut
[ERROR]   ITestAbfsFileSystemContractSecureDistCp>AbstractContractDistCpTest.testDistCpWithIterator:635 » TestTimedOut
[ERROR] Tests run: 271, Failures: 0, Errors: 2, Skipped: 40

NonHNS-SharedKey

[ERROR] Failures: 
[ERROR]   TestAbfsClientThrottlingAnalyzer.testManySuccessAndErrorsAndWaiting:171->fuzzyValidate:49 The actual value 14 is not within the expected range: [5.60, 8.40].
[ERROR] Tests run: 103, Failures: 1, Errors: 0, Skipped: 2
[WARNING] Tests run: 558, Failures: 0, Errors: 0, Skipped: 276 
[ERROR] Errors: 
[ERROR]   ITestAbfsFileSystemContractDistCp>AbstractContractDistCpTest.testDistCpWithIterator:635 » TestTimedOut
[ERROR]   ITestAbfsFileSystemContractSecureDistCp>AbstractContractDistCpTest.testDistCpWithIterator:635 » TestTimedOut
[ERROR] Tests run: 271, Failures: 0, Errors: 2, Skipped: 40

@steveloughran
Copy link
Contributor

steveloughran commented Aug 2, 2021

I've already merged the HADOOP-17811 retry one as it was painless; doing a distcp one next. The big troublespot is the paths one...can you provide that as a PR all on its own? thanks

@sumangala-patki
Copy link
Contributor Author

sumangala-patki commented Aug 2, 2021

I've already merged the HADOOP-17811 retry one as it was painless; doing a distcp one next. The big troublespot is the paths one...can you provide that as a PR all on its own? thanks

Sure! The paths one conflicts with the missing Correlation ID PR in 3.3, so that one (#2520) needs to go in first. Will raise backport PRs for both

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@sumangala-patki @hadoop-yetus @steveloughran @brianloss